Data networks are fundamentally a collection of data processing points, or network nodes, that are interconnected by communications paths between the nodes. There are many different types of networks and network data processing schemes. A given network may be characterized, for example, by the type of data transmission technology in use on it (for example, a Transmission Control Protocol/Internet Protocol (TCP/IP) or Systems Network Architecture network); by whether it carries voice, data, or both kinds of signals; by who can use the network (public or private); by the usual nature of its connections (dial-up or switched, dedicated or non-switched, or virtual connections); and by the types of physical links (for example, optical fiber, coaxial cable, twisted pair, etc.).
In many types of networks, data is passed along the communications paths in the form of packets. A packet is the unit of data that is routed between an origin and a destination on any other packet-switched network such as, for example, the Internet. When any file (e.g., e-mail message, hypertext markup language (HTML) file, Graphics Interchange Format (GIF) file, Uniform Resource Locator (URL) request, etc.) is sent from one place to another on a network, the Transmission Control Protocol (TCP) layer of the TCP/IP protocol divides the file into chunks of an efficient size for routing. Each of these packets is separately numbered and includes the Internet address of the destination so that the network nodes can efficiently keep track of, and route, the packets. The individual packets for a given file may travel different routes through the Internet. When they have all arrived, they are reassembled into the original file by the TCP layer at the receiving end.
Data traveling through a network may have various properties, or be characterized in a number of different ways. Data, for example, may be classified by “data flow” (i.e., a collection of related packets with similar packet properties) or “data conversation” (i.e., bi-directional packet flow). Furthermore, data may be characterized by whether it is moving in a “macro-flow” or a “micro-flow.” A micro-flow is a unidirectional signal content flow, whereas a macro-flow is made up of more than one unidirectional micro-flow. Division of macro- and micro-flows allows a hierarchical control and management of content flows. This hierarchical structure of traffic conditioners provides scalability to the collaborative streaming system architecture. In micro-flow traffic conditioners, data packets may be parsed at the object data packet level, thus enabling packet buffering, scheduling, and dropping to be carried out based on the object descriptor in the packet.
Packets traveling through packet networks may have also have associated “state” information. In general, the term “state” refers to the current or last-known status or condition of a process, transaction or setting. Therefore, “stateless” refers to data processing that does not keep track of configuration settings, transaction information or any other historical information about the processed data packets. When a processing program is stateless (i.e., it does not maintain state), or when the infrastructure of a system prevents a program from maintaining state, it cannot take information about the last session or packet into the next, such as settings the user made or conditions that arose during prior processing.
Stateless services do not maintain any “state” on a data flow or a data conversation basis and, therefore, do not correlate associated micro-flows together. Therefore, a processor will not know that two micro-flows of a TCP connection are associated, and similarly, for example, for data associated with an FTP program, control and data channels will not be correlated.
In contrast, stateful processing takes into account many conditions of the data, including historical packet activity. For example, some forms of network security applications require stateful processing. Such applications may require the monitoring, of conversations in the packet stream that comprise requests and responses to requests. Stateful processing permits the processor to keep track of the requests and responses by maintaining data about the history of packets.
Furthermore, stateful processing can correlate different bi-directional flows of multi-channel applications into a single, related conversation. For example, stateful processing can correlate bi-directional flows of the different 5-tuple bi-directional flows of FTP control and data into a single, related conversation since the Open Systems Integration (OSI) Model Layer 7 application data in the control channel packets explicitly refer to the 5-tuple of the data session flows.
In addition, state can be maintained across and outside of flow and conversations boundaries, for instances where Layer 7 data contained within packets is used to associate otherwise apparently unrelated flows or conversations. For example, state can be maintained in Hypertext Transfer Protocol (HTTP)/Secure Socket Layer (SSL) shopping card transaction transactions where 5-tuple conversations must be associated. State can also be maintained in SIP end-point communications that use different 5-tuple flows over time to refer to the same conversation by use of non-5-tuple conversation identification fields within the SIP application data.
Many network nodes process “packet flow” or “data flow” using a stateful processing method. Depending on the state of packets within a flow, rather than fully processing each packet, a node may fully process only a sampling of packets and determine how those packets should be handled. The node processor may them process similar packets (as determined by the packet's header information) in the data flow in the same manner without fully examining each packet.
Another use of statefully tracking flows and conversations is to be able to provide additional service to the application flows by understanding what the application packet flow is trying to perform by understanding and remembering the Layer 7 data contained within the packets; for example, intelligent security services, Quality of Service (“QoS”) and other functions can be applied to such data when the intent of the application is understood. Unfortunately, nodes that host large numbers of traffic flows have the potential to of significantly degraded service due to the demands of processing the data flows in this stateful manner.
There is a need, therefore, to determine when data may be processed statelessly, thereby saving system resources, and when data must be processed in a stateful manner due to the requirements of the processors and applications using the data (e.g., Firewalls, NAT, QoS, load-balancing applications, etc.) and the requirements of the data flow itself. There is a further need to provide implementation and management of stateful and stateless data processing. In addition, there is an identified need to manage stateful and stateless data processing at various levels of data processing. There is a further need to provide both manual and dynamic control of stateful and stateless data processing in order to provide greater control, and therefore greater efficiency, in data processing throughout data networks.